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1.  General  Summary  of  Achievements. 

The  purpose  of  the  grant  FA8655-05-1-3066,  “Multisensory  Mechanisms  of  Gaze 
Stabilization  and  Flight  Control“,  was  to  generally  support  research  in  my  lab  rather 
than  to  fund  a  specific  project.  At  the  time  the  grant  was  awarded  it  was  the  necessary 
condition  for  me  being  eligible  to  submit  a  research  proposal  to  the  BBSRC 
(Biotechnology  and  Biological  Sciences  Research  Council,  UK)  and  to  investigate, 
together  with  Prof  Simon  Laughlin  and  Dr  Graham  Taylor  (see  below)  the 
relationship  between  visual  sensor  axes  and  flight  modes  in  locusts  and  flies.  Soon 
after  the  BBSRC  grant  was  awarded  I  commenced  a  pennanent  position  in  the 
Department  of  Bioengineering,  Imperial  College,  UK,  which  allowed  me  to  use  the 
USAF  Research  Laboratory  funding  to  employ  an  additional  Research  Assistant  (RA 
=  postdoc)  in  my  lab. 

The  last  three  years  during  which  my  lab  was  funded  by  the  USAF  Research 
Laboratories  have  been  extraordinarily  productive.  Despite  the  fact  that  (a)  my  lab 


had  to  move  twice  -  from  Cambridge  to  temporary  space  at  Imperial  College  London 
and  then  to  our  new,  dedicated  rooms  -  and  (b)  that  I  was  appointed  Director  of 
Postgraduate  Studies  (Research)  being  responsible  for  about  70  PhD  students  in  the 
Department  of  Bioengineering,  the  overall  progress,  achievements  and  visibility  were 
highly  competitive  at  the  international  level.  This  is  reflected  by  a  couple  of 
performance  parameters  including  publications  (1.1),  WOS  Projects  (1.2),  Funding 
(1.3),  Visibility  (1.4),  and  Group  size  as  well  as  international  collaborations  (1.5). 

In  this  final  report  I  will  concentrate  on  the  performance  parameters  and  will  give  a 
brief  summary  of  the  scientific  progress  which,  to  a  great  extent,  is  covered  by  my 
previous  report  submitted  in  July  2008.  I  should  emphasise  that,  although  I  have  been 
receiving  funding  from  various  sources,  the  support  from  the  USAF  Research 
Laboratory  was  the  main  driving  force  in  the  progress  my  group  has  made  within  the 
last  three  years. 


1.1  Publications 

During  the  supported  period,  together  with  my  collaborators,  I  have  published  11  peer 
reviewed  papers,  some  in  high  profile  Journals  such  as  PLoS  Biology,  Current 
Biology,  and  Journal  of  Neuroscience,  one  comprehensive  book  chapter  on  central 
visual  processing  in  insects,  one  conference  proceeding,  and  11  meeting  abstracts. 

Bibliographic  Parameters 


Impact  of  publications  Number  of  outputs 


Additional  abstract  citations  =  28.  Data  based  on  Google  Scholar.  Snapshot  on  7.  December  2008 

For  more  specific  information,  please  see  my  personal  reference  list  at  the  end  of  this 
report. 


1.2  WOS  projects 

From  2003,  i.e.  two  years  before  the  funding  started,  until  2008,  I  have  applied  for 
and  was  awarded  6  WOS  grants.  These  projects  have  been,  without  exception, 
extremely  helpful  and  inspirational  (see  my  WOS  reports).  They  funded  the 
presentation  of  my  work  at  the  1st  and  2nd  workshop  on  compound  eyes,  University  of 


Arizona,  Tucson,  two  invited  talks  at  the  annual  meeting  of  the  Institute  of 
Navigation,  and  two  visits  to  the  AF  Research  Laboratory  in  Eglin,  FL.  The  last  visit 
to  the  Res  Lab  at  Eglin  Air  Force  Base  end  of  April  2008  was  particularly  exciting. 
On-base  briefings  on  two  days  gave  me  the  opportunity  to  meet  some  of  the 
contractors  working  in  areas  close  to  my  field  of  expertise.  At  one  of  the  meetings  I 
met  Dr  Sean  Humbert,  with  whom  I  currently  apply  for  a  USAF  Res  Lab  (AFOSR) 
project  grant  (see  below).  Another  outcome  of  my  last  WOS  visits  to  Eglin  AFB  was 
that  it  would  be  highly  desirable  to  institutionalize  my  annual  visits  to  the  Research 
Laboratory. 


1.3  Funding 

During  the  funded  period  I  have  written  and  submitted  a  number  of  grant  proposals, 
some  of  which  were  successful,  some  were  revised  and  resubmitted  and  some  will  be 
rewritten  and  sent  to  alternative  funding  agencies.  The  funding  secured  over  the  last  3 
years  is  in  excess  of  £1M  and  includes  a  grant  from  the  prestigious  HFSP  (Human 
Frontier  Science  Project)  trust. 

Funding  secured  for  research  in  my  laboratory  (excluding  FA8655-05-1-3066): 

•  BBSRC  research  collaboration  PI,  with  Prof  Simon  B.  Laughlin,  Co-PI 
University  of  Cambridge,  UK,  and  Dr  Graham  Taylor,  PI  Oxford  University, 
UK:  "Tuning  of  the  preferred  optic  flow  axes  of  locust  and  blowfly  visual 
interneurons  to  their  preferred  modes  of  flight  behaviour.  ” 

•  HFSP  research  collaboration  PI,  with  three  other  Pis:  Drs.  Manos  Drakakis, 
Imperial  College,  Fabrizio  Gabbiani,  Baylor  College  of  Medicine,  TX,  USA, 
Martin  Egelhaaf,  Bielefeld  University,  Germany:  “Comparative  analysis  of 
RF -transmitted  neural  activity  on  flying  insects.  ” 

•  Air  Force  Research  Laboratory/EOARD,  PI:  “ Multisensory  Control  of 
Stabilization  Reflexes.  “ 

•  Air  Force  Research  Laboratory/EOARD,  Co-PI,  with  Prof  Simon  B. 
Laughlin,  PI  University  of  Cambridge,  UK:  “ Multisensory  Integration  in  Fly 
Lobula  Plate  Tangential  Cells  I' 

Funding  secured  for  Department  of  Bioengineering  (PhD  programme): 

•  BBSRC  Doctoral  Training  Grant. 

Grant  proposals  under  consideration: 

•  AFOSR/NL,  through  EOARD,  PI,  together  with  Dr  Sean  Humbert,  PI 
University  of  Maryland:  “The  relationship  between  visual  sensor  equipment  in 
flying  insects  and  their  flight  performance  -  a  ‘Neurobio-Engineering’ 
approach 

•  EPSRC  (Engineering  and  Physical  Sciences  Research  Council,  UK)  research 
collaboration,  PI  together  with  three  other  Pis,  Drs.  Anil  Bharath,  Piere  Luigi 
Dragoti,  Manos  Drakakis,  Imperial  College,  UK:  “Biologically  inspired 
motion  processing  on  VLSI  devices.  ” 


Grant  proposals  in  preparation: 


•  BBSRC,  PI,  named  RA  Dr.  Martina  Wicklein.  This  grant  proposal  on 
“Neuronal  Basis  Underlying  Visual  distance  control”  was  turned  down  but 
will  be  revised  and  resubmitted  to  the  BBSRC.  We  propose  a  kinematic, 
dynamic  and  neurophysiological  analysis  of  the  hovering  behaviour  in 
hawkmoths. 

•  Framework  Programme  7,  European  Research  Council,  research  collaboration, 
PI,  together  with  Professor  Alessandro  Astolfi,  Dept.  EEE,  Imperial  College 
London,  Professor  Patrizio  Colaneri,  PI,  Dept,  of  Electronics  and  Information, 
University  of  Milano,  Italy,  Dr  Fritz  Lehmann,  PI,  Neurobiology,  University 
of  Ulm,  Gennany.  This  international  research  collaboration  is  in  its  pre¬ 
proposal  stage.  Building  up  on  previous  research  on  the  “Berkeley  Fly”  we 
propose  to  work  on  “Bio-inspired  Control  of  Micromechanical  Insects”  by 
integrating  the  expertise  of  scientists  working  on  biological  sensing  and 
control,  insect  aerodynamics,  and  control  engineering.  This  research 
collaboration  in  combination  with  the  proposal  submitted  to  AFOSR/NL 
should  yield  a  synergetic  effect  on  the  development  of  MAVs. 

•  EPSRC,  PI,  together  with  Dr.  Manos  Drakakis,  Dept  Bioengineering,  Imperial 
College,  on  the  development  of  bio-inspired  VLSI  sensors  for  optic  flow 
processing.  The  goal  is  a  generic  optic  flow  sensor  that  can  be  implemented  in 
a  wide  range  of  applications. 


1.4  Visibility 

Invited  talks:  from  2005  to  2008  I  have  been  invited  to  give  11  seminar  talks  as  well 
as  9 presentations  at  international  conferences/  workshops  (cf.  CV  for  details). 

Media  coverage:  Imperial  College  'News  and  Events'  interview/video-interview 
2008,  London  Times  interview  2008,  Imperial  College  press  release  on  latest 
publication  in  PLoS  Biology  2008,  Reporter  (Imperial  College)  2008,  Interview  “Die 
Zeit’ ’,  Gennany,  2008 


1.5  Group  size,  Collaborations,  and  Scientific  Impact 

Group  size:  Between  early  2005  (Cambridge)  and  end  2008  (Imperial  College)  my 
group  size  has  increased  from  2  postgraduate  students  to  currently  2  RA  and  3 
postgraduate  students.  In  addition  each  year  I  supervise  at  least  one  summer  student  as 
well  as  1-2  final  year  undergrad  or  MSc  projects. 

Collaborations:  Within  the  UK  I  currently  collaborate  with  groups  in  Cambridge, 
Oxford  and  groups  at  Imperial  College.  International  Collaborations  include  currently 
the  University  of  Bielefeld,  Gennany,  Baylor  College  of  Medicine,  USA,  and  the 
USAF  Research  Laboratory,  Eglin,  US).  Depending  on  the  outcome  of  my  recent 
grant  applications  further  collaborations  with  the  University  of  Maryland,  US,  the 


University  of  Ulm,  Germany,  and  the  University  of  Milano,  Italy,  will  be  added  (cf. 
1-3). 


Scientific  Impact:  Besides  the  publication-related  impact  (cf.  1.1),  I  regularly  review 
scientific  manuscript  for  19  different  Journals  including  Nature,  PLoS  Biology,  and 
Current  Biology.  I  am  also  acting  as  a  referee  for  national  and  international  grant 
agencies  (cf.  CV).  Finally,  I  was  invited  by  the  European  and  National  Science 
Foundations  to  assist  in  defining  the  grand  challenges  of  future  interdisciplinary 
research  collaborations  between  European  and  US  Biologists  and  Engineers  (cf.  CV.) 

I  was  recently  invited  to  give  a  presentation  at  a  workshop  organized  by  the  Gatsby 
Unit  for  Computational  Neuroscience,  University  College  London,  where  I  outlined 
the  research  in  my  lab.  This  international  “invitation  only”  workshop  on  “Simpler 
Cognitive  Systems”  resulted  in  plenty  of  positive  feedback  on  my  presentation  as  well 
as  an  invitation  by  the  editor  of  the  Journal  “Current  Biology”  to  write  about  ocellar 
systems.  Furthermore,  our  project  on  state-dependent  processing  using  octopamine 
agonists  (see  below)  was  also  very  well  received. 


2.  Report  on  current  projects: 

In  the  following  I  will  mainly  focus  the  latest  development  of  the  key  projects  (2.1- 
2.5).  More  detailed  information  about  background  and  relevance  of  the  projects  can 
be  found  in  my  previous  report  (July  2008).  Finally,  I  will  describe  two  further 
projects  (2.6,  2. 7),  which  have  been  or  will  be  started  shortly. 

2.1)  BBSRC  research  grant  collaboration:  Sensory  and  Motor  Control  Axes 

together  with  Dr  Graham  Taylor,  Department  of  Zoology,  Oxford  University. 

Project  started  in  July  2005  -  ended  in  September  2008 
Research  Associate:  Dr  Daniel  Wiistenberg. 

PI:  Holger  G  Krapp  (HGK),  Department  of  Bioengineering,  Imperial  College  London 
Co-PI:  Simon  B  Laughlin,  Department  of  Zoology,  University  of  Cambridge 

This  project  was  based  on  the  hypothesis  that  the  sensor  axes  along  which  flying 
insects  measure  self-motion  parameters  are  aligned  with  those  motor  axes  the  animals 
use  to  control  their  specific  (aerodynamic)  modes  of  flight  (024).  To  test  the  “mode¬ 
sensing”  hypothesis,  we  combined  quantitative  behavioural  data  (Oxford  team)  with 
the  preferred  rotation  axes  of  optic  flow-processing  interneurons  in  flies  and  locusts 
obtained  from  electrophysiological  experiments  (A31). 

I  will  attach  here  the  final  overall  report  on  this  BBSRC  project  which  provides  a 
summary  of  our  achievements  and  will  only  briefly  describe  the  electrophysiological 
results  obtained  in  my  lab  at  Imperial  College. 

Major  electrophysiological  finding:  Recordings  from  directional-selective  cells  in  the 
locust  lobula  were  significantly  more  difficult  to  achieve  than  those  from  lobula  plate 
tangential  cells  (LPTCs)  known  to  measure  optic  flow  parameters  (06,  024,  R3).  We 
nevertheless  succeeded  in  studying  a  set  of  wide  field  neurons  in  locust  by  means  of 
mainly  extracellular  and  a  few  intracellular  recordings  (A31).  Locusts,  like  flies, 
employ  directional-selective  cells  with  extended  receptive  fields.  The  distribution  of 


local  motion  preferences  in  these  receptive  fields  suggests  that  they  are  tuned  to 
certain  self-motions  which  indicates  they  play  a  similar  role  in  visually  controlled 
behaviour  as  the  fly  lobula  plate  tangential  cells  (cf.  Fig.  1,  attached  BBSRC  report). 
A  detailed  analysis  of  the  receptive  field  with  respect  to  the  preferred  self-motion  axes 
in  locust  shows,  however,  that  there  are  marked  differences  between  the  two  species 
(cf.  Fig.  4,  attached  BBSRC  report):  (i)  While  flies  employ  a  comparatively  large 
number  of  LPTCs,  10  VS  cells,  to  sense  horizontal  body  rotations,  locusts  -  in  tenns 
of  rotations  -  mostly  analyse  turns  around  slightly  tilted  vertical  body  axes,  (ii)  Flies 
do  not  have  LPTCs  which  cover  forward  translation  directly,  but  locusts  do. 

The  overall  analysis  of  all  experimental  data  obtained  in  this  project  has  not  been 
completed  yet.  At  this  point  in  time,  therefore,  we  can  not  draw  any  firm  conclusions. 
The  experimental  evidence  however  suggests  that  not  only  in  dipteran  flies  but  also  in 
orthopteran  locusts  are  directional-selective  wide-field  cells  tuned  to  specific  optic 
flow  fields.  This  suggests  a  general  functional  principle  in  visuo-motor  control.  The 
differences  in  preferred  self-motion  axes  analysed  by  the  flies  and  locusts  seem  to 
reflect  the  species-specific  adaptations  to  the  animals’  respective  aerodynamic 
properties  and  flight  modes.  Locusts  endowed  with  a  higher  degree  of  passive 
aerodynamic  stability  than  flies  apparently  employ  a  smaller  number  of  pitch-  and 
roll-sensitive  cells  than  the  aerodynamically  unstable  but  more  manoeuvrable  fly. 

Although  this  project  has  come  to  an  end  I  will  continue  the  collaboration  with  Dr 
Graham  Taylor’s  group  to  further  investigate  the  functional  relationship  between 
sensors  and  actuators  in  insect  flight  based  on  quantitative  behavioural  data  on  insect 
aerodynamics.  A  scientific  publication  of  the  electrophysiological  data  is  currently  in 
planning. 

This  project  is  an  essential  building  block  for  the  collaborative  grant  proposal  together 
with  Dr.  Sean  Humbert  (cf.  1.3)  submitted  to  AFOSR/NL  on  a  comparative  study  of 
sensors  (compound  eye/ocelli)  and  flight  control  in  robber  flies  and  horse  flies  (cf. 
proposal  No:  09NL-002).  My  previous  report  in  combination  with  the  proposal 
contains  further  information  about  the  relevance  of  this  project  in  the  context  of  the 
control  of  autonomous  (micro)  air  vehicles. 


2.2)  Research  Associate  project  on:  Neck  Motor  Neurons 

Funding  from  the  Medical  Research  Council  (MRC)  to  Dr.  Stephen  Huston  and 

from  USAF  grant  to  HGK. 

Supervisor:  HGK 

The  experimental  part  of  this  project  on  the  neuronal  pathway  involved  in  fly  gaze 
stabilization  has  (temporarily)  come  to  an  end  as  Dr.  Stephen  Huston  is  now  a  postdoc 
at  Caltech,  USA.  Stephen  characterized  the  response  properties  of  fly  neck  motor 
neurons,  in  particular  their  specificity  to  certain  optic  flow  fields.  His  contribution 
was  the  first  to  show  in  detail  how  self-motion  information  provided  by  LPTCs  is 
used  in  the  well-defined  neck  motor  system  that  controls  compensatory  head 
movements. 


After  the  first  part  of  Dr.  Stephen  Huston’s  work  has  now  been  published  (026)  we 
are  currently  preparing  a  second  publication  on  the  non-linear  interaction  between 


haltere  and  visual  input  at  the  level  of  the  neck  motor  neurons.  Stephen  found  that 
some  neck  motor  neurons  set  up  an  “AND  gate”  that  requires  both  visual  and  haltere 
(gyroscopic)  input  before  the  neurons  generate  action  potentials  which  are  a  necessary 
condition  to  control  neck  muscle  contraction  (cf.  last  report,  028,  A33).  As  I 
indicated  in  my  previous  report  such  gating  mechanism  may  be  an  adaption  to  the 
kinematic  properties  of  different  muscle  blocks  controlling  slow,  intennediate  and  fast 
head  movements.  To  test  this  hypothesis  we  have  started  another  PhD  project  on  the 
functional  organization  of  the  neck  motor  system  (cf.  project  2.6,  below). 

Background  and  relevance  of  this  project  are  described  in  my  previous  report.  Here,  I 
would  like  to  emphasise  the  importance  of  head  movements  for  the  analysis  of  optic 
flow  and  the  significance  of  fast  sensorimotor  pathways  in  the  context  of  flight 
control.  Head  movements  are  a  necessary  prerequisite  for  optic  flow-based  flight 
control  as  they  stabilize  a  default  orientation  of  the  visual  environment  and  thus 
simplify  the  extraction  of  self-motion  information.  For  this  to  work,  head  movements 
need  to  cover  an  extended  dynamic  range  which,  in  terms  of  sensor  measurements, 
involves  several  complementary  mechanisms  (multisensor  fusion).  Compound  eyes, 
ocelli  and  halteres  provide  information  on  attitude  changes,  but  at  different  delays  and 
for  different  sections  of  the  overall  dynamic  input  range.  The  neck  motor  system  is  a 
perfect  model  system  to  study  several  issues  in  control  engineering:  (i)  dynamic  range 
fractionation,  (ii)  multisensor  fusion,  and  (iii)  sensorimotor  feedback/feed  forward 
control.  Furthermore,  image  stabilization  is  a  fundamental  issue  in  vision-based 
pattern  matching  (e.g.  seekers). 


2.3)  PhD  project  on:  Multimodal  Modulation  of  LPTCs 

Funding  from  the  Biological  and  Behavioural  Sciences  Research  Council  (BBSRC)  to 
Mr  Matthew  Parsons  and  from  USOSR/NL  to  Prof.  Simon  Laughlin. 

Supervisors:  Simon  B.  Laughlin,  HGK  (co-supervisor). 

Two  years  ago  Matthew  Parsons  demonstrated  for  the  first  time  that  the  activity  of  an 
identified  LPTC,  the  spiking  VI  cell,  is  modulated  by  ocellar  input  (020,  A32).  His 
PhD  thesis  is  a  milestone  in  the  research  on  multisensory  integration.  Matthew  has 
now  passed  his  viva  and  will  continue  his  work  in  Dr  Simon  Laughlin’ s  lab  and  under 
my  co-supervision  on  the  integration  of  ocellar/compound  eye  signals  for  the  next  1 1 
months  (funded  by  USAFOSR/NL,  cf.  1.3). 

In  my  last  report  I  presented  Matthew’s  data  on  intracellular  recordings  from  VS  cells. 
Matthew  successfully  recorded  from  8  out  of  10  VS  cells,  all  of  which  received  input 
from  the  ocelli.  Because  the  spiking  VI  cell,  which  he  recorded  from  in  his  first  sets 
of  experiments,  are  known  to  receive  input  from  VS1-VS3,  it  was  to  be  expected  that 
the  activity  in  these  cells  could  be  modulated  by  ocellar  input.  It  was  not  clear, 
however  that  all  the  other  VS  cells  also  received  ocellar  input.  Matthew  developed  a 
quantitative  ocellar  stimulus  which  allowed  him  to  mimic  rotations  of  the  fly  around 
various  horizontal  body  axes.  Applying  this  stimulus  during  intracellular  recordings 
he  found  that  each  VS  cell  has  a  preferred  axis  tuning,  the  shape  of  which  is  similar  to 
that  mediated  by  wide-field  optic  flow  stimulation  of  the  compound  eye.  Further  data 
analysis  showed  that  the  distribution  of  ocellar-mediated  preferred  rotation  axes 
seems  to  reflect  the  cardinal  measuring  axes  the  ocellar  system  used  to  sense  the  fly’s 
changes  in  attitude,  i.e.  rotations  around  the  roll  axis  and  an  axis  intermediate  between 


pitch  and  roll.  Apparently,  the  ocellar  system  also  signals  horizontal  body  rotations, 
but  at  a  much  coarser  resolution  than  the  compound  eye.  Matthew  also  performed 
pioneering  behavioural  experiments  using  a  newly  developed  apparatus  that 
stimulates  both  the  compound  eyes  and  the  ocelli.  His  first  results  indicate  that  the 
faster  ocelli  responses  advance  the  signals  mediated  by  the  compound  eye  and  thus 
speed  up  the  processing  of  attitude  changes  in  the  VS  cells. 

A  more  comprehensive  description  of  the  current  findings,  their  functional 
implication,  and  future  research  can  be  found  in  the  proposal  on  “Multisensory 
Integration  in  Fly  Lobula  Plate  Tangential  Cells”  submitted  by  Prof  Simon  Laughlin 
to  AFOSR/NL  through  EOARD.  We  are  currently  working  on  a  scientific  publication 
of  Matthew’s  latest  electrophysiological  results  and  will  set  up  for  further  behavioural 
and  electrophysiological  studies. 


2.4)  PhD  projects  on:  Sensory  Processing,  Multisensory  Integration,  and  Efference 
Copies  in  Behaving  Animals 

Funding  from  Department  of  Bioengineering,  Imperial  College,  and  the  USAF  to 
HGK  (supplementary  overseas  student  fees).  PhD  Students:  a)  Mr  Naveed  Ejaz,  b)  Mr 
Kris  Peterson.  Collaborator:  Dr.  Emmanuel  Drakakis,  Dept.  Bioengineering,  Imperial 
College  London. 

PI:  HGK. 

The  ultimate  goal  of  this  project  is  to  build  a  closed-loop  experimental  robot  platform 
that  will  allow  us  to  investigate  multisensory  integration  in  a  closed-loop  system. 
During  the  last  6  months  the  PhD  students  working  on  this  project  made  significant 
progress  in  that  they  succeeded  in  closing  the  loop  between  neural  activity  and  robot 
actuation/steering.  This  intermediate  step  along  the  project  was  necessary  to  show  that 
it  is  actually  possible  to  use  neuronal  activity,  action  potential,  of  an  identified  LPTC, 
HI,  to  control  a  robot. 

The  set  up  the  students  use  still  consists  of  a  conventional  electrophysiology  rig  and 
two  high  speed  (200  fps)  computer  displays  for  visual  stimulation.  The  fly  is 
positioned  in  front  of  the  displays,  one  positioned  in  the  left,  the  other  one  in  the  right 
visual  hemisphere  at  +/-  45  degrees  relative  to  the  animal’s  longitudinal  body  axis. 
External  to  the  rig  a  robot,  equipped  with  two  high  speed  video  cameras,  is  positioned 
on  a  turn  table  that  is  moved  clockwise  at  slightly  variable  angular  speed.  The  robot  is 
surrounded  by  a  pattern  that  consists  of  vertical  strips,  which,  because  of  the 
movement  of  the  turntable,  induces  wide-field  motion  that  is  picked  up  by  the 
cameras  and  transmitted  via  thin  wires  to  two  PCs.  The  PCs  output  the  pattern  motion 
to  the  computer  displays  mentioned  above.  Due  to  the  pattern  motion,  the  spiking 
activity  of  an  extracellularly  recorded  HI  cell  is  modulated  to  a  degree  that  depends 
on  pattern  velocity.  The  recorded  spikes  are  identified  by  a  threshold  algorithm  and 
are  converted  into  a  continuous  voltage  signal  that  is  fed  back,  via  wireless  Bluetooth 
transmission,  to  the  RISC  processor  of  the  robot.  During  the  first  experiments  the 
control  algorithm  was  chosen  so  that  the  robot  motor  system  compensates  for  the 
externally  forced  movement  of  the  turn  table. 

With  this  simple  “fly-machine  interface”  operating  under  closed  loop  conditions  we 
could  show  that  neuronal  HI  signals  are  sufficient  to  perfonn  an  image  stabilization 


task.  Our  first  results  using  the  interface  have  the  following  significant  implications 
for  further  research: 


a)  Proof  of  concept:  the  experimental  approach  chosen  to  study  multisensory 
integration  under  closed-loop  conditions  is  realistic.  In  the  final  configuration, 
i.e.  with  the  fly  mounted  on  the  robot,  it  will  allow  us  to  assess  the 
contributions  of  various  sensory  systems  to  motor  control  while  the  fly  is 
actually  moving  in  space. 

b)  Optimizing  control  algorithms  in  a  closed-loop  brain  machine  interface:  this  is 
a  more  general  implication  which  addresses  a  fundamental  issue  of  brain- 
machine  interfaces.  Because  the  sensory  motor  pathway  in  the  fly  is 
sufficiently  well  know  with  respect  to  its  sensory  input  we  can  now  study  and 
optimize  control  algorithms  which  take  into  account  some  of  the  fundamental 
properties  of  biological  sensor  system  such  as  neuronal  noise,  neuronal 
adaptation,  and  pattem/illumination-dependent  sensor  signal.  The  goal  will  be 
to  develop  control  algorithms  which  are  independent  of  visual  stimulus 
parameters  and  thus  allow  for  pattern-invariant  feedback  control. 

c)  Characterization  of  biological  sensor  systems:  So  far  the  fly  visual  system  has 
been  studied  by  focussing  on  its  coding  properties  -  which  did  not  explicitly 
take  into  account  the  link  to  the  various  motor  systems.  Our  new  experimental 
platform  allows  us  to  characterize  visual  information  processing  within  a  solid 
control-engineering  framework  that  takes  into  account  the  closed  loop  nature 
of  a  biological  control  system  as  well  as  its  plasticity,  adaptability,  and  the 
potential  use  of  forward  models  (efference  copy). 

Early  next  year  we  will  be  featuring  the  new  system  internally  at  Imperial  College 
before  presenting  it  at  the  Gottingen  Neurobiology  Conference,  Germany,  2009,  and 
preparing  a  scientific  publication  -  probably  in  one  of  the  IEEE  journals.  With  this 
project  I  finally  bridged  the  gap  between  basic  neurobiological  research  and  control 
engineering.  The  potential  applications  go  far  beyond  the  analysis  of  the  fly  visuo- 
motor  pathways  per  se  and  will  probably  have  an  impact  on  the  development  of 
various  technical  and  medical  applications  where  brain-machine  interfaces  are 
required. 

The  layout  of  the  silicon  micro  recording  probe  that  will  be  implanted  in  the  fly  head 
capsule  to  pick  up  signals  from  spiking  LPTCs  has  now  been  finalized  and  will  be 
sent  out  for  production  early  January.  We  expect  to  have  the  first  prototype  for  testing 
in  the  lab  by  April  2009. 


2.5)  Research  Associate  project  on:  State-dependent  Processing  in  LPTCs. 

Funding  from  the  USAF  to  HGK.  Research  Associate:  Dr  Kristopher  Longden. 

PI:  HGK. 

Dr  Kit  Longden  has  continued  his  work  on  state-dependent  processing  of  optic  flow 
using  pharmacological  drugs  to  switch  the  fly’s  brain  from  resting/walking  to  flight 
state  (A32).  The  follow-up  grant  from  the  USAF  Research  Laboratory  (FA8655-09-1- 
3022)  has  now  been  activated.  Together  with  Mr  Daniel  Schwyn,  PhD  student  (see 
below),  Kit  set  up  a  fly  colony  which  will  provide  a  constant  supply  of  experimental 


animals  for  the  lab.  He  also  continues  to  be  the  integrating  figure  in  my  lab  by  helping 
to  supervise  various  projects  and  taking  on  organizational  duties. 

I  have  been  reporting  about  progress  of  his  research  in  my  last  report  (July  2008). 
Here  I  should  confine  myself  to  his  latest  achievements  and  new  collaborations  he 
initiated  with  the  Bio-sensor  groups  in  our  Department. 

By  applying  the  octopamine  agonist  Chordimeform  (CDM)  -  octopamine  is  a 
neurotransmitter,  neuromodulator  and  neurohonnone  in  insect  nervous  systems  -  Kit 
induces  a  Active  flight  state  in  flies.  Besides  its  impact  on  neural  processing, 
octopamine  also  switches  metabolic  pathways,  i.e.  during  walking  and  resting 
carbohydrates  are  metabolized  for  energy  supply  of  muscles  and  neurons  while  during 
flight  fatty  acids  are  being  used.  He  then  studies  the  changes  in  neuronal  processing 
compared  to  the  processing  in  the  same  animals  before  treatment.  His  major  findings 
are: 

a)  The  spontaneous  activity  of  spiking  LPTCs,  VI  (roll-pitch  sensor)  and  V2 
(roll  sensor),  is  significantly  elevated  after  drug  application.  This  effectively 
changes  the  set  point  of  the  LPTCs  and  allows  the  cells  to  exploit  the  negative 
and  positive  dynamic  response  range  in  a  more  efficient  way. 

b)  The  response  delay  in  both  V 1  and  V2  is  reduced. 

c)  The  information  rate  transmitted  by  V 1  and  V2  is  increased. 

Altogether,  these  changes  can  be  interpreted  as  short  term  adaptations  to  the  different 
dynamic  input  ranges  flies  encounter  when  walking/resting  or  flying,  given  energy 
constraints  (Laughlin  2001).  Less  energy  supply  to  the  nervous  system  and  the  (leg) 
muscles  is  required  during  walking/resting.  When  airborne,  however,  the  flight  motor 
consumes  considerably  higher  amounts  of  energy  and  the  visual  system,  including  the 
LPTCs,  process  information  at  higher  bandwidth  which  is  energetically  more 
expensive.  Because  power  constraints  present  a  fundamental  issue  for  micro  air 
vehicles  with  limited  payload,  Kit’s  research  on  energy  efficient  coding  is  highly 
relevant  for  the  development  of  low  power,  bio-inspired  sensorimotor  control  systems 
(for  further  information  on  this  project  see  my  July  2008  report). 

To  include  more  experimental  results  and  the  infonnation  theoretical  analysis  in  Kit’s 
first  publication  we  had  to  postpone  manuscript  submission  by  a  few  months.  The 
changes  on  the  manuscript  are  now  nearly  completed  and  we  expect  its  submission  in 
January  2009. 

Kit  also  started  to  collaborate  with  the  Bio-sensor  groups  of  Drs.  Danny  O’Hare  and 
Martyn  Boutelle  in  the  Department  of  Bioengineering  at  Imperial  College.  These 
groups  have  the  instrumentation  to  measure  octopamine  concentrations  in 
haemolymphe  samples  taken  from  the  fly  during  different  locomotor  states.  In 
collaboration  with  the  Bio-sensor  groups,  we  are  also  aiming  to  measure  the  changes 
of  adenosine-triphosphate,  a  molecule  that  provides  energy  for  any  muscle  and 
neuronal  activity.  The  idea  is  to  establish  the  correlation  between  neuronal  processing 
and  energy  consumption  during  different  locomotor  states. 


2.6)  PhD  project:  Characterization  of  Head  Movements  in  the  Blowfly. 


Building  up  on  previous  studies  in  the  laboratories  of  Drs.  Strausfeld  and 
Hengstenberg  (rev.:  R3)  and  detailed  electrophysiological  experiments  in  my  own  lab 
(cf.  2.2,  026),  Daniel  Schwyn,  a  PhD  student,  is  setting  up  the  instrumentation  for  a 
functional  characterization  of  the  blowfly  gaze  stabilization  system.  Daniel  has 
completed  a  first  prototype  of  the  LED  arena  he  will  be  using  for  visual  stimulation  in 
behavioural  experiments.  He  will  perform  both  a  dynamic  and  a  kinematic  analysis  of 
head  movements  during  visual  stimulation  -  which  will  be  eventually  combined  with 
electrophysiological  recordings  from  LPTCs.  He  set  up  and  tested  our  6  DoF  force 
transducers  as  well  as  a  classic  yaw  torque  meter  for  dynamic  measurements.  While 
experiencing  technical  difficulties  with  the  6  DoF  force  transducer  (faulty  channels), 
Daniel  has  successfully  implemented  the  torque  meter  which  will  be  also  be  used  in 
collaborative  and  student  projects. 

To  complement  the  characterization  of  the  fly  neck  motor  system  an  MSc  student  will 
perform  electrical  stimulations  of  neck  muscles  and  neck  motor  neurons  while 
monitoring  head  movements  using  high  speed  video  cameras. 

Understanding  the  functional  organization  of  the  neck  motor  system  is  an  important 
prerequisite  for  any  biologically  inspired  gaze  stabilization  systems  which  facilitates 
the  use  of  optic  flow  for  flight  control. 


2. 7)  BBSRC proposal:  Visual  Distance  Control. 

I  reported  on  this  proposal  in  my  previous  report  (July  2008).  We  proposed  a 
dynamic,  kinematic  and  electrophysiological  study  of  visual  distance  control  in 
hovering  hawkmoths.  By  integrating  the  results  of  this  multi-level  analysis  we  should 
derive  a  comprehensive  model  of  a  robust  outer-loop  collision  avoidance  system. 
Unfortunately,  the  proposal  was  unsuccessful  when  submitted  for  the  first  time. 
Because  of  the  relevance  of  this  project  in  the  context  of  outer-loop  flight  control  I 
decided  to  include  work  on  hawkmoths  into  the  HFSP  collaboration  which  has  now 
started.  The  HFSP-funded  project  is  aiming  to  develop  micro  recording  probes  which 
include  an  RF  module  for  wireless  transmission  of  neuronal  signals  and  therefore  goes 
beyond  the  scope  of  the  project  Kris  Peterson  (2.4)  is  working  on.  The  goal  of  the 
HFSP  study  will  be  to  record  neural  activity  in  semi-  or  freely-flying  locusts  and  flies. 

I  will  also  revise  and  resubmit  the  proposal  on  visual  distance  control  to  the  BBSRC, 
or  to  other  funding  agencies,  as  soon  as  we  will  have  obtained  the  first  preliminary 
results.  Dr  Martina  Wicklein  will  start  her  work  in  January  2009. 


Concluding  remarks 

Over  the  last  3  years  we  have  established  an  integrative  System  Neuroscience 
approach  in  my  lab  that  hugely  benefits  from  the  interdisciplinary  environment  at 
Imperial  College  Fondon.  Funding  from  several  sources  -  mainly  from  the  USAF  -, 
access  to  excellent  students  and  the  close  proximity  and  collaboration  with  engineers 
and  neuroscientists  have  had  a  significant  impact  on  the  progress  we  were  able  to 
achieve  so  far.  Further  collaborations  with  engineers  at  Imperial  College,  the  US  and 


in  Italy  will  help  me  to  establish  a  true  bridge  between  basic  neuroscience  research 
and  bio-inspired  control  engineering. 


Holger  G  Krapp 


London,  UK,  15.  Dec  2008 
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